On Gaussian Expectation Propagation
نویسنده
چکیده
In this short note we will re-derive the Gaussian expectation propagation (Gaussian EP) algorithm as presented in Minka (2001) and demonstrate an application of Gaussian EP to approximate multi-dimensional truncated Gaussians. 1 On Gaussian Distributions Here we will summarise some important equalities about the Gaussian density. A Gaussian density in Rn is defined by N (x;μ,6) := (2π)− 2 |6|− 1 2 exp ( − 2 (x− μ)T6−1 (x− μ) ) . (1.1) We will write x ∼ N (x;μ,6) to both denote x has a distribution P(x) and that the density of this distribution is given by (1.1). We will write N (x) as a shorthand for N (x; 0, I). For t ∈ R, we will denote the cumulative Gaussian distribution function by 8(t;μ, σ 2) which is defined by 8 ( t;μ, σ 2 ) = Px∼N (x;μ,σ 2) (x ≤ t) = ∫ t −∞ N ( x;μ, σ 2 ) dx . (1.2) Again, we write 8(t) as a shorthand for 8(t; 0, 1). We will write 〈 f (x)〉x∼P to denote the expectation of f over the random draw of x , that is 〈 f (x)〉x∼P := ∫ f (x)d P(x). The following results are given without proof; for a detailed derivation the reader is referred to Herbrich (2002). Linear transformation x ∼ N (x;μ,6) and y = Ax+ b⇒ y ∼ N ( y;Aμ+ b,A6AT ) . Convolutions Assume x ∼ N (x;μ,6) and y|x ∼ N (y;Ax,0) . Then x|y ∼ N ( x;9 ( AT0−1y+6−1μ ) ,9 ) , (1.3) y ∼ N ( x;Aμ,0 + A6AT ) , (1.4) where 9 := (AT0−1A+6−1)−1. Marginals Let us assume that a random vector x is composed such that x ∼ N ([ x1 x2 ] ; [ μ1 μ2 ] , [ 611 612 612 622 ]) .
منابع مشابه
Improving posterior marginal approximations in latent Gaussian models
We consider the problem of correcting the posterior marginal approximations computed by expectation propagation and Laplace approximation in latent Gaussian models and propose correction methods that are similar in spirit to the Laplace approximation of Tierney and Kadane (1986). We show that in the case of sparse Gaussian models, the computational complexity of expectation propagation can be m...
متن کاملGaussian Process Regression with Censored Data Using Expectation Propagation
Censoring is a typical problem of data gathering and recording. Specialized techniques are needed to deal with censored (regression) data. Gaussian processes are Bayesian nonparametric models that provide state-of-the-art performance in regression tasks. In this paper we propose an extension of Gaussian process regression models to data in which some observations are subject to censoring. Since...
متن کاملLoop corrections for message passing algorithms in continuous variable models
In this paper we derive the equations for Loop Corrected Belief Propagation on a continuous variable Gaussian model. Using the exactness of the averages for belief propagation for Gaussian models, a different way of obtaining the covariances is found, based on Belief Propagation on cavity graphs. We discuss the relation of this loop correction algorithm to Expectation Propagation algorithms for...
متن کاملApproximate Marginals in Latent Gaussian Models
We consider the problem of improving the Gaussian approximate posterior marginals computed by expectation propagation and the Laplace method in latent Gaussian models and propose methods that are similar in spirit to the Laplace approximation of Tierney and Kadane (1986). We show that in the case of sparse Gaussian models, the computational complexity of expectation propagation can be made comp...
متن کاملOn numerical approximation schemes for expectation propagation
Several numerical approximation strategies for the expectation-propagation algorithm are studied in the context of large-scale learning: the Laplace method, a faster variant of it, Gaussian quadrature, and a deterministic version of variational sampling (i.e., combining quadrature with variational approximation). Experiments in training linear binary classifiers show that the expectation-propag...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2005